Inline updates for HMMs
نویسندگان
چکیده
Most training algorithms for HMMs assume that the whole batch of observation sequences is given ahead of time. This is particularly the case for the the standard EM algorithm. However, in many applications such as speech, the data is generated by a temporal process. Singer and Warmuth developed online updates for HMMs that process a single observation sequence in each update. In this paper we take this approach one step further and develop an inline update for training HMMs. Now the parameters are updated after processing a single symbol of the current observation sequence. The methodology for deriving the online and the new inline update is quite different from the standard EM motivation. We show experimentally on speech data that even when all observation sequences are available (batch mode), then the online update converges faster than the batch update, and the inline update converges even faster. The standard batch EM update exhibits the slowest convergence.
منابع مشابه
Covariance updates for discriminative training by constrained line search
We investigate the recent Constrained Line Search algorithm for discriminative training of HMMs and propose an alternative formula for variance update. We compare the method to standard techniques on a phone recognition task.
متن کاملHidden Gauss-Markov models for signal classification
Continuous-state hidden Markov models (CS-HMMs) are developed as a tool for signal classification. Analogs of the Baum, Viterbi, and Baum–Welch algorithms are formulated for this class of models. The CS-HMM algorithms are then specialized to hidden Gauss–Markov models (HGMMs) with linear Gaussian state-transition and output densities. A new Gaussian refactorization lemma is used to show that th...
متن کاملHidden Markov Models: Applications to Flash Memory data and Hospital Arrival times
A hidden Markov model (HMM) is a bivariate Markov chain which encodes information about the evolution of a time series. HMMs can faithfully represent workloads for discrete time processes and therefore be used as portable benchmarks to explain and predict the complex behaviour of these processes. This project introduces the main concepts of HMMs for discrete time series including a summary of H...
متن کاملIncremental MAP estimation of HMMs for efficient training and improved performance
Continuous density observation hidden Markov models (CD-HMMs) have been shown to perform better than their discrete counterparts. However, because the observation distribution is usually represented with a mixture of multi-variate normal densities, the training time for a CD-HMM can be prohibitively long. This paper presents a new approach to speed-up the convergence of CD-HMM training using a ...
متن کاملMAN-MACHINE INTERACTION SYSTEM FOR SUBJECT INDEPENDENT SIGN LANGUAGE RECOGNITION USING FUZZY HIDDEN MARKOV MODEL
Sign language recognition has spawned more and more interest in human–computer interaction society. The major challenge that SLR recognition faces now is developing methods that will scale well with increasing vocabulary size with a limited set of training data for the signer independent application. The automatic SLR based on hidden Markov models (HMMs) is very sensitive to gesture's shape inf...
متن کامل